Repository navigation
Conversation
Background jobs used to be started in three different ways: a builtin `go` statement, `KVStore.Go`, and `KVTxn.spawnWithStorePool`. Each of them tracked (or failed to track) the goroutine differently, which is the kind of mismatch that caused pingcap/tidb#57213. Make `KVStore.Go` the single entry point. It now registers the job with the store WaitGroup, runs it in the store pool, and rejects new jobs once the store starts closing, so `Close` waits for background jobs instead of racing with them. `KVTxn.spawn` and `KVTxn.spawnWithStorePool` collapse into one `spawn` that goes through it. Along the way: - `Spool.Run` returns `ErrPoolClosed` instead of letting the underlying pool silently drop the function, so a rejected job is no longer indistinguishable from an accepted one. `Spool.Close` is now idempotent. - `KVTxn.spawn` calls the `Post` lifecycle hook when the goroutine is rejected, which keeps it balanced with the `Pre` hook that already ran. - Callers that prepared state for a background job (`cleanWg`, the async pessimistic rollback WaitGroup, the broadcast rate limit) release it when the job is rejected, instead of leaving a waiter blocked forever. - `WaitGroup()` is dropped from the `kvstore` interface used by the transaction package and deprecated on `KVStore`. Foreground fan-out goroutines that the caller joins before returning, such as `batchExecutor.startWorker` and `KVStore.updateMinSafeTS`, are left as plain `go` statements: they are not background jobs and do not outlive their caller. Closes tikv#1510 Signed-off-by: ljluestc <63439129+ljluestc@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @ljluestc. Thanks for your PR. I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Welcome @ljluestc! |
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Background jobs used to be started in three different ways: a builtin
gostatement,KVStore.Go, andKVTxn.spawnWithStorePool. Each of them tracked (or failed to track) the goroutine differently, which is the kind of mismatch that caused pingcap/tidb#57213.Make
KVStore.Gothe single entry point. It now registers the job with the store WaitGroup, runs it in the store pool, and rejects new jobs once the store starts closing, soClosewaits for background jobs instead of racing with them.KVTxn.spawnandKVTxn.spawnWithStorePoolcollapse into onespawnthat goes through it.Along the way:
Spool.RunreturnsErrPoolClosedinstead of letting the underlying pool silently drop the function, so a rejected job is no longer indistinguishable from an accepted one.Spool.Closeis now idempotent.KVTxn.spawncalls thePostlifecycle hook when the goroutine is rejected, which keeps it balanced with thePrehook that already ran.cleanWg, the async pessimistic rollback WaitGroup, the broadcast rate limit) release it when the job is rejected, instead of leaving a waiter blocked forever.WaitGroup()is dropped from thekvstoreinterface used by the transaction package and deprecated onKVStore.Foreground fan-out goroutines that the caller joins before returning, such as
batchExecutor.startWorkerandKVStore.updateMinSafeTS, are left as plaingostatements: they are not background jobs and do not outlive their caller.Closes #1510